Musical instruments are manipulated by a musician-user to articulate and produce notes of discrete tones having constant pitch. For example, a guitar has strings and a finger board with frets that divide the guitar strings into discrete lengths. By pressing a string against a fret with a finger, a musician can select equally spaced pitches. Each musical note need not be articulated separately. Some notes may be played merely by changing the pitch of a previously articulated note, and pitch may frequently be altered to glide continuously between notes.
To more fully appreciate the present invention, it is helpful to define several terms commonly used by professional musicians. "Pitch" is the subjective sensation produced by a periodic vibration having constant frequency, e.g., what may be produced by a picked (or plucked) guitar string that is under tension. The sensation of pitch is logarithmically related to the frequency of vibration. In music, discrete pitches are referred to as "notes".
The "tone" or "timbre" of a musical note includes several components, and gives the note a distinctive character. The same note played on a organ will have a different timbre than if played on a piano. Indeed, the same note played on two pianos can exhibit different nuances of timbre, and will sound different. The timbre of an instrument playing a note is closely related to the shape of the periodic wave creating the note, which is referred to as waveshape. As such, one aspect of timbre is the manner in which the note waveshape changes with time. Timbre is often analyzed in terms of the instantaneous Fourier series components of a tone. Ordinarily these components change over time, corresponding to the change in waveshape. There are also non-harmonic components of timbre including, for example, breath noise admixed with the simple, round tone of a flute.
A listener can distinguish different instrumental timbres, such as a note from an oboe and a note from a violin. A critical aspect enabling the differentiation is the complex change that occurs within the first few milliseconds of the onset of the note (known as the "attack" time). This aspect is related to how notes are produced (or "articulated") on different instruments. For example, a violinist articulates notes by bowing, to produce a scraping sound and a sense of the note growing from nothing, whereas a saxophonist tongues notes.
Tone synthesizers use control envelopes to simulate the interactions of pitch, volume and timbre. Control envelopes may be characterized by four associated parameters, namely attack, decay, sustain and release.
The "distance" in pitch between two notes is termed the musical "interval". In the familiar "do-re-mi" scale, each syllable represents an interval or distance from the first note of the scale. The creation of many musical effects involves altering the pitch of a previously articulated note, and may require production in a manner different than the articulation of musical notes. Musical effects can include slurs, hammer-ons, pull-offs, blues inflections, glissandos, portamento, and vibrato. Such gestures are usually performed to move from one note of a scale to an adjacent note, or in between, the movement being referred to as the distance of a gesture.
"Blues inflection" is the musical term for a voice-like pitch bend gesture, and usually includes "blue note" pitches that are located between pitches found in standard musical scales. Most popular American music incorporates blues-type note inflections in some way, and thus a mechanism for bending notes is necessary for an electronic synthesizer used to play such music.
Professional grade musical synthesizers routinely provide some sort of pitch bend mechanism for performing continuous pitch changes, such as over a major second interval as might be produced by a guitar player by bending strings. The most common type of pitch altering device is the pitch wheel. The pitch wheel may be a user-operated biased rotary wheel with a center detente, or a rotary wheel with a dead zone in the middle. Other control mechanisms for altering pitch include spring-biased levers and joysticks. Unfortunately, such mechanisms do not generally produce lifelike pitch inflections.
On a guitar, blues inflection is performed by deflecting a guitar string sideways (laterally) along the fret after the string has been picked (e.g., caused to vibrate) by the musician. This lateral movement increases the tension on the vibrating string, and hence increases the pitch. Similarly, a skilled saxophonist may bend pitch using air pressure or lip pressure on the playing reed.
A "slide" is performed on a guitar by sliding the finger up or down the finger-board (longitudinally), after an initial note is picked. A similar effect, referred to as "glissando", may be produced on a piano by dragging a thumb back and forth across the keys while pressing down, to cause each key to sound quickly in succession. The result is a series of additional discrete sound pitches. On a harp, such a glissando may be performed by dragging the hand across the strings, so that each sounds in succession. A similar gesture on a guitar is called a "strum", one difference being that the notes which sound on a guitar are each spaced by several semitones, making a chord. On a harp, chords may be produced by using pedals to retune the strings.
When the hand is then dragged across the strings as when performing a glissando, the result is called an "arpeggio". Apreggios usually traverse a wide range of dozens of notes, while a guitar strum is always six notes or less. A guitar chord may also be "plucked" as may a chord on a harp, while a chord on a piano is "struck". Arpeggios may also be played on a piano by striking the notes of a chord in succession while alternately displacing each hand to cover a wide range of notes. However, this produces a significantly different effect than a harp arpeggio because it requires discrete motions of the fingers used to strike each note.
Some musical notes may be played by slurring a previously articulated note. "Slurring" means a musician does not produce every note anew but may instead continue from one note to the next without re-attacking, thus changing only the pitch. For example, on a saxophone, notes are slurred by opening or closing additional valves on the body of the instrument, while continuously blowing. This changes the pitch and creates a different note, but without a tongued articulation. On a violin, slurring involves placing fingers in front of or behind other fingers on the fingerboard while continuing to bow, to shorten or lengthen the effective vibrating length of the string. This changes the pitch and creates a different note, but without a bowed articulation. Guitarists slur notes similarly as violinists, but the gesture of placing a first finger on a fret in front of a second finger is called a "hammer-on". The gesture of placing a second finger behind the first and then releasing the first is called a "pull-off". Playing whole phrases by articulating only the first note and slurring the remaining notes is called "legato".
In real life, gestures may be combined, either sequentially or simultaneously. Thus a skilled guitarist can perform a hammer-on followed by a glissando, or simultaneously achieve blues inflection and vibrato. A number of hammer-ons in quick succession produces a "trill". A series of connected gestures traversing a large interval in a single direction is called a "run". A combination of gestures involving one or more changes of direction may be referred to as a "lick" or a "riff". These gestures are so named because each is perceivable as an individual musical event. Their separate elements become fused because there is an overarching time shape to them. This time shape or trajectory may result from a continuous change in the times between expected onset of the component gestures.
In addition, guitarists and other musicians may perform gestures within gestures. That is, they may alter the apparent course of a gesture as it is performed by making slight variations. Such subtleties may distinguish one player's style from another, or even a great performance from a poor one.
Electronic musical instruments such as synthesizers are known in the art, and often include various sound generating routines that are executed by a digital signal processor, or the like. Such synthesizers are realized using digital oscillators to produce tones using cyclic time vectors and wavetables. Typically a mechanism is provided for selecting discrete pitches, such as a piano-like keyboard, and at least one mechanism for altering pitch continuously. Historically, such pitch altering or bending mechanisms have been primarily mechanical in nature, and in addition to pitch wheels and joysticks mentioned above, included levers, ribbons, and various modifications to piano keyboards. But such prior art pitch altering mechanisms permit only a single type of gesture, and the sounds they produce do not include very expressive gestures. In general, they do not permit combining gestures, or alteration of the gesture as it is performed.
In addition, no means of strumming chords is provided. A strum may be simulated on a piano-like keyboard by staggering the notes of a chord as they are struck. However, this does not produce a realistic sounding strum because it requires coordinating several discrete gestures by the player's fingers. A guitar strum, or harp glissando is executed by a single motion with the arm.
Gesture mapping gloves and touch responsive membranes have been used to try to provide more flexible pitch alteration. But even these more complex mechanisms produce sounds that are less expressive than desired because acoustic instruments (and the human voice) require some exertion to produce musically useful results. It is known to also provide exertion-requiring mechanism using force-feedback based on linear motors or high-tension springs. But such mechanisms are expensive, bulky, and difficult to use, and require special circuitry to interface with electronic sound-production systems. They also suffer from the same limitations as standard mechanical operators.
At best, the prior art has attempted to bend pitch in a single direction. Thus with respect to producing the sound of a plucked guitar string, one can attempt to emulate a guitarist's transverse deflection (blues inflection) of a plucked string, or perhaps a lateral deflection (glissando), but not both transverse and lateral gestures. Interestingly, the prior art sometimes suggests that pitch is a non-critical parameter with which to impart expression in one direction. By implication, the prior art would regard as futile attempts to bend pitch in more than one direction.
It is known in the prior art to assist the simulation of gestures using electronics. Thus, some synthesizers create slurs using a so-called "mono mode" operating state in which only one note may be played at a time. If a second note is played before releasing the previous note, the second note continues the first note with only a change in pitch. Further, the envelopes used to create the first note continue rather than beginning anew from the attack. If a note is played after all previous notes have been released, the new note is re-attacked. Unfortunately, mono mode does not create effective slurs because on real musical instruments pitch changes that create slurs do not occur suddenly, but have a characteristic pitch change curve. In U.S. Pat. No. 5,216,189, to Kato (1993), this problem was somewhat addressed using preset curves to create the slur, where the above-described fingering scheme was retained.
"Portamento" or "pitch glide" is another gesture, in which a continuous gliding movement from one musical tone to another is produced without rearticulation. For example, a trombonist moves the trombone slide in and out while continuing to blow; a violinist slides one finger up or down the fingerboard while continuing to bow. (Technically, the gesture that creates portamento on a violin creates a glissando on a guitar because a guitar has frets that cause discrete pitches to be produced as each fret is crossed, whereas pitch varies continuously for a violin.)
Many attempts have been made in the prior art to create portamento on electronic sound synthesizers, typically using a pre-programmed function. In some synthesizers, portamento effect circuitry is activated to automatically cause each note to slide to the next note over a period of time. In some implementations, so-called fingered portamento is created by playing the second note before releasing the first, analogously to slurring. Functions used to create such synthesized portamento or pitch glide typically are characterized by an exponential curve. While such curves produce a recognizable effect peculiar to synthesizers, they do not realistically duplicate natural sounding portamentos performed on actual musical instruments.
One device called the Oberheim Strummer was designed to produce realistic guitar strums for use with electronic instruments. This device had prerecorded strum templates activated by pressing a button. While such an implementation produces realistic single strums, they always sound the same. The speed of the strum can not be altered, nor varied as it is performed.
Another device known in the art is referred to as an arpeggiator. This works, as the name suggests, by activating the notes of a chord held down on a piano-like keyboard rapidly in succesion. The notes are evenly spaced and therefore do not have the overarching time characteristic that gives real arpeggios their lifelike perceptual quality. The result is an automated computer music effect.
Some synthesizers produce portamento by actually performing the gesture using a ribbon controller comprising a position-sensing membrane whose length may be a few inches to three feet or so. While such configurations can produce relatively realistic portamentos, the portamento is of a single characteristic type. U.S. Pat. No. 5,241,126 to Usa et al. (1993) discloses the use of transfer functions transitioning from ribbon position to pitch, to try to produce a higher quality portamento. Often the transfer function has a stair-like step characteristic, that upon performance seeks to produce a glissando-like gesture.
"Vibrato" is the undulating variation that creates tension when sustaining a single tone for a period of time, and may be produced by most bowed-string and woodwind classical instruments. On prior art synthesizers, vibrato may be implemented using a cyclic low frequency waveform that automatically varies (or frequency modulates) the pitch of a note. The amount of frequency modulation produced may be controlled with a continuous modulation wheel, and may be developed gradually using a control envelope or some other mechanism, such as a pedal or a pressure pad. Unfortunately the vibrato effect produced inevitably sounds automatic, and not lifelike.
Some prior art synthesizers include tables of pre-drawn transfer function curves, but do not provide a means of altering the predrawn-curves during performance. For instance, U.S. Pat. No. 5,241,126 to Usa et al.(1993), referred to above, seeks to bridge the gap between mechanical operators and electronic tone generation with a mechanism that uses a continuous operator along which gestures can be mapped. Unfortunately, Usa's disclosed mechanical operator is sophisticated mechanically and thus expensive to manufacture. Significantly, Usa fails to adequately specify means of simulating a variety of realistic gesture performance techniques. The few gesture maps specified are simplistic and representative of easily modeled gestures, e.g. using step functions to perform scales, arpeggios, or glissandos. Unfortunately, the output from Usa's device is not MIDI-compatible, and cannot be exported for use by other MIDI-compatible systems.
MIDI refers to a standard data communications protocol for electronic music equipment. In the MIDI specification, bytes of data are coded to be recognized as representing musical functions such as note on and note off of specific keys on a piano keyboard. Certain bytes are coded and specified to represent ranges of digital numbers for use modifying continuously variable parameters of musical tones. Ranges of numbers are specified for pitch bend and overall volume. All MIDI synthesizers recognize these numbers, and most recognize additional numbers specified for general use, that may be used to modify synthesizer functions controlling various aspects of timbre.
Having discussed the nature of various musical effects that are desired from a musical synthesizer, it is now useful to examine how such synthesizers are configured. FIG. 1 is a generic block diagram of a prior art musical synthesizer 10, and is similar to the commercially available Ensoniq model SD-1. Synthesizer 10 includes a discrete pitch generator 20 that outputs chosen discrete pitches in response to the output from a user input signal, typically provided by a piano keyboard 30. Each discrete pitch is presented by generator 20 as a digital number.
Synthesizer 10 further includes an interface module 40 that contains menu-selectable programming parameters, and is responsive to user-input values from controls 50, 50' (respectively "OP1" and "OP2"), which may include wheels, foot pedals, a pressure pad, or the like. It is a function of module 40 to modify tone synthesis parameters for use by tone synthesizer 60. At best, however, what results is an unrealistic pitch bend sound that is quite unlike the pitch inflection actually produced by a musician using a traditional musical instrument.
Module 40 further includes a scaler 70 whose output is a scaled digital number proportional to the output from the first operator control 50. This user-determined digital ouput number is then input to a look-up table 80 that typically performs a waveshaping and amplitude normalizing function. The result from look-up table 80 is then combined by a adder 90 with a digital number representing the output from the second user control 50'.
Summer 90 outputs a digital number representing the combined effects of operator controls 50 and 50'. The partial sum output by summer 90 is then combined by a second adder 100 with the digital number representing the output of the discrete pitch generator 20. It will be appreciated that adder 100 may represent any means of combining discrete pitch data from discrete pitch generator 20, and continuous pitch data from adder 90, including merging of MIDI data within tone generator 60. The grand output provided by adder 100 is a digital number or numbers that can control the output of tone generator 60. As shown in FIG. 1, synthesizer system 10 may instead provide the output from the first adder 90 as an input to the tone generator 60, without digital summation using adder 100. In either embodiment, the output from tone synthesizer 60 is input to an audio reproduction system, e.g., loudspeaker 110.
Unfortunately the musical effects provided by prior art synthesizer 10 are not especially realistic. At best there is a crude attempt to bend pitch. For example the scaler 70 cannot effectively control musical intervals because the parameters specified are not scale-tone divisions. Further, synthesizer 10 cannot provide effectively controlled musical intervals because scaler 70 precedes (rather than follows) look-up table 80. At best, scaler 70 can affect, but in an unpredictable fashion, the extent of waveshaping by the lookup table 80. Further, there is no means provided of modifying gesture trajectory in real time.
There is a need for a gesture synthesizer permitting realistic simulations of various musical gestures in any sequence for an electronic musical instrument, including digital synthesizers. Preferably such a gesture synthesizer should realistically alter pitch, tone volume, and tone timbre. Further, such a gesture synthesizer should be modular in use and in fabrication, and should be implemented using conventional components and techniques. Finally, pitch bend and other gestures provided by such a gesture synthesizer should be MIDI-compatible and MIDI-exportable.
The present invention discloses such a gesture synthesizer.